VR-Doh: Hands-on 3D Modeling in Virtual Reality
ACM Transactions on Graphics (SIGGRAPH), 2025
Zhaofeng Luo*1 2, Zhitong Cui*1 3, Shijian Luo3, Mengyu Chu2 4, Minchen Li1
1 Carnegie Mellon University 2 Peking University 3 Zhejiang University 4 State Key Lab of General AI
* Equal Contribution
Abstract
We introduce VR-Doh, a hands-on 3D modeling system that enables intuitive creation and manipulation of elastoplastic objects in Virtual Reality (VR). By customizing the Material Point Method (MPM) for real-time simulation of hand-induced large deformations and enhancing 3D Gaussian Splatting for seamless rendering, VR-Doh provides an interactive and immersive 3D modeling experience. Users can naturally sculpt, deform, and edit objects through both contact- and gesture-based hand-object interactions. To achieve real-time performance, our system incorporates localized simulation techniques, particle-level collision handling, and the decoupling of physical and appearance representations, ensuring smooth and responsive interactions. VR-Doh supports both object creation and editing, enabling diverse modeling tasks such as designing food items, characters, and interlocking structures, all resulting in simulation-ready assets. A user study with both novice and experienced participants highlights the system's intuitive design, immersive feedback, and creative potential. Compared to existing geometric modeling tools, VR-Doh offers enhanced accessibility and natural interaction, making it a powerful tool for creative exploration in VR.
Modeling from scratch
Click on an image to explore the 3D model in the viewer
3D GS Object Editing Examples
Click on an image to explore the 3D GS in the viewer
Input 3D GS
Edited 3D GS
UI Walkthrough
@article{luo2025vrdoh,
author = {Zhaofeng Luo and Zhitong Cui and Shijian Luo and Mengyu Chu and Minchen Li},
title = {VR-Doh: Hands-on 3D Modeling in Virtual Reality},
year = {2025},
issue_date = {July 2025},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
volume = {44},
number = {4},
issn = {0730-0301},
url = {https://doi.org/10.1145/3731154},
doi = {10.1145/3731154},
journal = {ACM Trans. Graph.},
month = july,
numpages = {12},
keywords = {Virtual Reality, 3D Modeling, Elastoplasticity Simulation, Material Point Method, Human-Computer Interaction}
}